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Considered by many as a source of valuable data and a potential to improve mathematics 
education, a significant amount of studies have been conducted on The National Assessment 
Program — Literacy and Numeracy (NAPLAN) test results. This study is a systematic literature 
review of 86 peer-reviewed journal articles published between 2008 and 2017 to investigate 
how NAPLAN numeracy test results were used in those studies. Findings showed NAPLAN 
results were used primarily to map student progress and identify strengths and weaknesses in 
teaching. 


The National Assessment Program — Literacy and Numeracy (NAPLAN) is an annual 
assessment for of Australian students in Years 3, 5, 7 and 9, undertaken since 2008. This 
standardised test assesses students' reading, writing, language (spelling, grammar, and 
punctuation) and numeracy administered by the Australian Curriculum, Assessment and 
Reporting Authority (ACARA). The Federal Government pushes such assessment to achieve 
public accountability, demonstrate transparency, and maintain public confidence in the 
standards of the education system throughout Australia (Klenowski & Wyatt-Smith, 2012). 

The NAPLAN test plays a key role in establishing and raising standards of learning (e.g., 
Hardy, 2014; Polesel, Dulfer & Turnbull, 2012). It is assumed that NAPLAN test results create 
opportunities for thoughtful dialogue and discussion to improve teaching and learning 
practices. In this regard, ACARA (2017) identified various areas to use NAPLAN test results 
for teachers, parents, schools and government bodies. For example, NAPLAN test results could 
be used to help teachers to challenge higher performers and identify students needing support. 
For parents, the NAPLAN test results supply individual student level reports to enable parents 
to see their child’s progress over the course of their schooling. It also provides each school 
aggregated data to identify strengths and weaknesses within their teaching programs (Polesel, 
Dulfer & Turnbull, 2012). According to ACARA (2017), the NAPLAN test results can be used 
to; 

¢ Challenge higher performers and identify students needing support 

¢ Map student progress, identify strengths and weaknesses in teaching and set goals. 

¢ Discuss progress with teachers and compare performance against national peers. 

¢ Support good teaching and learning, and school improvement. 

The present study used a systematic literature review of 86 peer-reviewed journal articles 
which focused on NAPLAN numeracy test results. The four purposes of the NAPLAN test 
results were categorised for the systematic literature review. The study investigates how 
NAPLAN numeracy test results were used in the reviewed journal articles in relation to the 
four purposes of NAPLAN results listed by ACARA (2017). The study focuses only on the 
numeracy test results. As a result, the study is guided by two research questions. Firstly, which 
purposes for the NAPLAN results are focused on in studies? Secondly, what are the gaps in 
using NAPLAN numeracy test results? The contribution of this study lies in the procedures 
used to review the articles in particular to mathematics education research, the use of 
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mathematics test results to inform practice in mathematics education and identify gaps to 
inform future mathematics education studies. 


Background 


Since 2008, the NAPLAN test results are available and reported as a mean scale score 
compared to the national minimum standard (such as the skills and understandings students 
can demonstrate at their particular year of schooling, in a specific subject area or domain) 
(ACARA, 2017). The report is also available to be selected by gender, indigenous status, 
language background, geolocation, parental occupation and parental education at each year 
level and for each domain (reading, writing, spelling, grammar and punctuation, and numeracy) 
of the test (ACARA, 2017). 

Results from NAPLAN test have a number of potential uses. They can be used to monitor 
the performance of the education system, inform classroom practice, ensure that students have 
met required educational standards and encourage teacher and schools for their students’ 
performance (ACARA, 2017; Rosenkvist, 2010). The NAPLAN test results can also provide 
schools with data to analyse and sense trends occurring in schools that can inform planning and 
policy decisions (Perso, 2009). According to ACARA (2017), the NAPLAN results can be used 
for four purposes. 

Firstly, the NAPLAN test results can be used to challenge higher performers and identify 
students needing support (ACARA, 2017). In this regard, there is a considerable body of 
research literature (e.g., Nichols and Berliner 2007; Stobart 2008; Taubman 2009; Darling- 
Hammond 2010) cited in (Lingard & Sellar, 2013, p.634) demonstrating the effects of 
standardised testing results to inform teachers’ pedagogical practices and improve students 
learning outcomes. Therefore, individual NAPLAN results can support teachers to plan for 
individual student improvement (Perso, 2009). 

Secondly, the NAPLAN test results provide useful information to map student progress, 
identify strengths and weaknesses in teaching and set future goals (ACARA, 2017). The 
NAPLAN test results are available in aggregated forms at the national and school level. As a 
result, schools can gain detailed information from NAPLAN test results about how they are 
performing and identify strengths and weaknesses which may lead to further attention and 
interventions. To identify strengths and weakness of students numeracy competency for future 
intervention, Hardy (2014) urged that the validity of the NAPLAN test is considered (measures 
of students’ actual learning of mathematics) and to ensure well-understood measures of 
students’ achievement. 

Thirdly, the NAPLAN test results are good source of information to discuss students’ 
progress and compare their performance against national peers (ACARA, 2017). This 
comparison and the reported outcomes of the test enable the Australian public to develop a 
general national perspective on student achievement and, more specifically, an understanding 
of how their child and schools are performing in relation to the national standards (ACARA, 
2017). Such data are assumed to develop confidence in Australians that education resources 
are allocated to ensure that all students achieve meaningful learning during their time at school 
(Guenther, 2013). 

Finally, at the system level, the NAPLAN test results provide education ministers with 
information about the success of their policies and resourcing in priority curriculum areas 
(ACARA, 2017). It also provides ministers with the capacity to monitor the success of policies 
aimed at improving the achievement of different student groups, such as indigenous students. 
Such data provide an additional suite of information, thus enhancing the capacity for evidence- 
based decision making about policy, resourcing and systemic practices at the system level 
(Klenowski & Wyatt-Smith, 2012). 
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In addition to the four purposes, the NAPLAN test results have become a powerful tool to 
describe education systems, assess teaching quality and determine school funding formulae 
(Guenther, 2013). 

A significant number of studies have been conducted on NAPLAN and used NAPLAN 
results in their reports for various purposes. Hardy (2014) used NAPLAN numeracy results as 
useful data for grouping students to help improve their numeracy capabilities, and as a stimulus 
for teacher professional development. Burrows, Goldman, Olson, Byrne, and Coventry (2017) 
used NAPLAN test results to show the impact of increased consumption of sugar-sweetened 
beverages on numeracy test scores and suggested strategies to improve the students’ numeracy 
competency. 

This study used a systematic literature review of peer-reviewed journal articles published 
between 2008 and 2017 which focused on NAPLAN numeracy test results. The study 
investigated how NAPLAN numeracy test results were used in these studies in relation to the 
four purposes of NAPLAN numeracy test results. 


Method 


This study used a systematic literature review. The search was conducted in five scientific 
databases (i.e., Education Resources Information Center (ERIC), Web of Science, Scopus, 
Science Direct, and Academic Search Complete). The general search terms for all databases 
included the Boolean operators ‘AND’ and the wildcard (*) function. To limit the scope of the 
study, the review was limited to peer-reviewed articles published between 2008 and 2017, and 
full-text availability was required. In addition, studies focusing either only NAPLAN numeracy 
or both of NAPLAN literacy and numeracy were used in the review. When the studies were on 
both numeracy and literacy aspect, only the numeracy aspect was considered for further review. 

Similar to the suggestion by Cronin, Ryan, and Coughlan (2008), this study followed the 
steps shown in Figure 1 to conduct the systematic review. The review passed through 5 steps 
from identifying the search term to categorising search results according to their focus. 


*({NAPLAN* AND Numeracy*] 
Indentify 


search term 


*ERIC 

-Academic search complete 
Indentify * Scopus 

Scientific * Science Direct 

Database 


search results 


* Studies used NAPLAN results in their report 


ees Pe ° Studies focus included numeracy results 


journal articles 


* Challenge higher performers and identify students needing support 

+ Map student progress, identify strengths and weaknesses in teaching programs and set goals 
Catagorise * Discuss progress with teachers and compare performance against national peers 

BEE «Support good teaching and learning, and school improvement 


Rass -Reults in Spreedsheets 
repititive 


Figure 1. Database search and review process. 
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The initial search yielded 284 studies. However, further screening (availability of the full 
report, relevant to NAPLAN test results and removing repetitive search results) resulted in 160 
journal articles. 

Additional screening (studies used NAPLAN results in their report and studies focusing on 
NAPLAN numeracy test results) revealed 86 peer review journal articles. The search result 
across each search engine and publication year is shown in Table 1. All the studies used the 
quantitative NAPLAN numeracy test results (mean scores) in their reports. 

In the end, these journal articles were reviewed and categorised according to the four 
purpose criteria (ACARA, 2017) mentioned in the background section. 


Table | 
Relevant Search Results for Review 
Year published 
Search Source 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total 
Scopus 0 0 1 2; a a ) 8 8 34 
Science Direct 0 0 0 2 0 5 7 7 12 33 
ERIC 4 1 1 2 3 0 1 0 0 12 
Academic Search complete 0 0 0 0 0 0 0 6 0 6 
Web of Science 0 0 0 0 0 0 0 0 1 1 
Total 4 1 2 6 8 10 13 21 21 86 


In order to get a more comprehensive understanding, articles were further categorised 
according to year level (Years 3, 7, 9 or only primary, secondary or both or no year level focus) 
and type of school (government or non-government [independent, Catholic] or both or not 
mentioned). A spreadsheet was used to document, extract information about the categories and 
analyse the data. 


Results and Discussion 


As shown in Table 2, seven studies (8.1%) used NAPLAN numeracy test results in their 
report to identify higher performers and students needing support. For example, Perso (2009) 
analysed NAPLAN test items and indicated that students need to be taught how to deal with 
the literacy demands of a task if students are to become numerate. 


Table 2 


Purpose and Year Level 


Year published 
Purpose 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total 


Map student progress, identify strengths 0 0 1 0 0 3 4 4 13. 25 (29.1%) 
and weaknesses in teaching and set goals 

Support good teaching and learning, and 0 0 0 1 1 1 5 5 3. 16 (18.6%) 
school improvement 

Discuss progress with teachers and 0 0 1 2 1 2 2 2 2 15(17.4%) 
compare performance against national 

peers 

Challenge higher performers and identify 2 0 0 0 0 1 0 2 2 7 (8.1%) 
students needing support 

Others 2 1 0 23 (26.7%) 


Ww 
an 
Ww 
N 
Nn 
— 


Total 4 1 2 6 8 10) 13) 2121 86 
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A large number of studies (N = 25 (29.1%) used NAPLAN test results in their report to 
map student progress, identify strengths and weaknesses in teaching and set goals. Using 
multilevel modelling to account for within-school variables, Chua, Khan, Humphry, and 
Hassell (2017) analysed NAPLAN test results to estimate the effect of the national partnerships 
on student performance. The results indicated that on average male students performed higher 
in the numeracy test. Vetter, O’connor, O’Dwyer, and Orr (2015) argued the importance of 
fitness for general numeracy competency. Similarly, Burrows et al. (2017) showed that 
increased consumption of sugar-sweetened beverages was associated with significantly lower 
test scores in numeracy. To identify students needing support for their numeracy competency, 
Brew, Toelle, Webb, Almqvist, and Marks (2014) used NAPLAN test results to investigate the 
effect of omega-3 fatty acid supplementation on subsequent numeracy performance in children. 
All these studies used NAPLAN test results to identify the possible reason for students’ weak 
achievement in numeracy with little suggestion on the possible strategies to improve their 
numeracy competency. 

The search results revealed sixteen studies (18.6%) focusing their report on supporting 
good teaching, learning, and school improvement. Hardy (2015, p.335)’s research showed 
schools that dominated with high numeracy results indicated good teaching practice and 
demonstrated the schools’ focus on the students as ‘valued capitals.’ Similarly, Polesel, Dulfer, 
and Turnbull (2012) demonstrated that NAPLAN test results are used in schools as a source of 
information to plan intervention and engaging in curriculum development practices to improve 
teaching and learning in the school. 

Fifteen (17.4%) studies focused on NAPLAN results to map student progress and compare 
performance against national peers. Ford (2013) analysed the inequality of achievement 
between indigenous and non-indigenous students in the States and Territories, with particular 
reference to New South Wales and the Northern Territory. In relation to comparing students 
against national peers, Marks (2016) used NAPLAN results to show the relative effects of 
socio-economic, demographic, non-cognitive and cognitive influences on student achievement 
in Australia. Others used NAPLAN results to compare national peers and suggest improved 
strategies for teachers in relation to mobile learning (Males, Bate, & Macnish, 2017), disability 
(Teather & Hillman, 2017), class size (Watson, Handal, & Maher, 2016) and NAPLAN scores. 

There were a large number of studies (N = 23 [26.7%]), in the search result, which used 
NAPLAN numeracy test results in their report, with a different focus than those listed by 
ACARA (2017). For example, a study by Quinnell and Carter (2013) draws the reader’s 
attention to the large variety of symbols, abbreviations, and conventions used in the NAPLAN 
numeracy tests. Norton (2009) provided a critique of the Year 9 NAPLAN numeracy test and 
how results might inform teaching mathematics. Norton (2010) used Year 9 NAPLAN 
numeracy test results to examine pre-service teachers’ mathematics content knowledge. 
Rogers, Barblett, and Robinson (2016) investigated the impact of NAPLAN numeracy tests on 
student, parent and teacher emotional distress in independent schools. This result showed that 
the NAPLAN test results were not limited to the four purposes listed by ACARA (2017). 

The review also considered the trends of these studies across publication year as shown in 
Figure 2. Studies published toward 2017 have an increased focus on supporting good teaching 
and learning through analysing test results. 
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-5 2009 2010 2011 2012 2013 2014 2015 2016 2017 


Pupulication year 


Number of studies 
Nn 


—=General 
=== Support good teaching and learning, and school improvement 

Discuss progress with teachers and compare performance against national peers 
=== Map student progress, identify strengths and weaknesses in teaching and set goals 


=== Challenge higher performers and identify students needing support 
Figure 2. Trends of using NAPLAN results from 2009 to 2017. 


As shown in Table 3, studies conducted at primary schools (N = 15), focused on their report 
to map student progress, identify strengths and weaknesses in teaching. Whereas, a significant 
number of studies were conducted combining both primary (Years 3 and 5) and secondary 
(Years 7 and 9) school NAPLAN numeracy test results (N = 37) and their primary focus was 
diverse (such as mathematics teachers perception on NAPLAN testing). 


Table 3 
Trends in the Studies Across Year Levels 
Year level 
Purpose 
3 5 7 9 Primary Secondary Both Total 
Map student progress, identify strengths 5 2 2 1 4 4 8 25 
and weaknesses in teaching and set goals 
Support good teaching and learning, and 3 1 0 1 4 3 4 16 
school improvement 
Discuss progress with teachers and 2 0 0 0 1 3 9 15 
compare performance against national 
peers 
Challenge higher performers and identify 1 1 0 0 4 0 1 7 
students needing support 
Others 1 0 0 0 2 4 15 22 
Total 12 4 2 2 15 14 37 86 


The search result showed that most of these studies were conducted with a combination of 
government and non —government schools (N = 40[46.5%]). Twenty-five studies (29%) were 
conducted in government schools, and eight were conducted in non — government schools. The 
remaining studies (N = 13[15.1%]) didn’t explicitly mention the type of schools included in 
their report. 


Conclusion 


The purpose of this review was to investigate how NAPLAN numeracy test results were 
used in the 86 peer-reviewed journal articles in relation to the four purposes of NAPLAN 
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results listed by ACARA (2017). From the review results, NAPLAN numeracy test results were 
used for various purposes in the reviewed studies. However, as this systematic review showed, 
a large number of studies used NAPLAN test results to map student progress, identify strengths 
and weaknesses in teaching. Hardy (2014) showed the importance of using NAPLAN test 
results as a source of evidence for grouping students to help improve their numeracy 
capabilities. Au (2013) examined the impact of childhood obesity on academic performance 
and identified children requiring support to improve their numeracy competency. Smith et al. 
(2014) grouped students as breakfast skippers and non-breakfast skippers impacting their 
NAPLAN scores on which breakfast skippers scored lower NAPLAN scores in numeracy. 
These studies identified possible reasons for students’ performance, with limited 
recommendations for relevant interventions to improve students numeracy competency. 

Interestingly, a significant number of studies used NAPLAN numeracy test results different 
from the four purposes provided by ACARA (2017). For example, Tayler et al. (2016) showed 
the importance of NAPLAN scores to provide evidence on how best to invest in Early 
Childhood Education and Care. Males, Bate and Macnish (2017) studied the impact of mobile 
learning on students NAPLAN scores. These studies show that the NAPLAN numeracy test 
results can be analysed for a wide variety of purposes, not limited to the four purposes provided 
by ACARA (2017). In addition, these studies (Males, Bate & Macnish, 2017; Tayler et al., 
2016), used well-thought-out, valid research to evidence the impact of various interventions, 
(e.g., using technologies) on students’ performance in numeracy. 

A limited number of studies used the NAPLAN numeracy results to challenge higher 
performers and identify students needing support. It was the expectation of this study that 
majority of the reviewed articles would focus on this purpose. Polesel, Dulfer & Turnbull 
(2012) warn that the focus on NAPLAN test results has shifted the culture of some schools to 
‘teach to the test’ and identify students needing support prior to sitting NAPLAN. . It is the 
contention of this study that studies identifying students needing support will support schools 
to plan interventions for students who are in most need with the intention of increasing success 
in their numeracy. In this regards, Hardy (2015) suggested that identifying high or low 
NAPLAN tests scores of students is an essential element in the provision of support and design 
interventions. 

The authors recommend the following directions for future research on NAPLAN 
numeracy test results. First, even though a large number of studies used NAPLAN test results 
to map student progress, identify strengths and weaknesses in teaching, none of these 
questioned the validity of the NAPLAN tests. Future studies focusing on the validity of the 
NAPLAN tests are fundamental to use the results for multiple purposes. In this regard, Hardy 
(2014) recommended the importance of valid tests to inform future planning and intervention. 
Secondly, as the review results elicited, a limited number of studies used NAPLAN numeracy 
test results to identify students’ weakness and suggest possible strategies to improve the result. 
Future studies could focus on identifying students’ weakness in a specific mathematics branch 
such as algebra, measurement, geometry, probability or statistics. This could support teachers, 
schools, and policymakers to plan relevant interventions at national, school and individual 
student level. Finally, future studies should focus on assisting teachers with data analysis of 
NAPLAN numeracy results (such as item analysis, using item analysis report) to evaluate 
students’ performance and plan their teaching and learning programs. 

This study is limited to a review of peer-reviewed journal articles and a few search engines. 
Broader inclusion of publications (such as conference papers, and books) and a range of search 
engines could provide a more compressive picture about the use of NAPLAN numeracy test 
results. However, this study could be used as a starting point for similar studies. 
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